Data Mining in Astronomical Databases

A Virtual Observatory (VO) will enable transparent and efficient access, search, retrieval, and visualization of data across multiple data repositories, which are generally heterogeneous and distributed. Aspects of data mining that apply to a variety of science user scenarios with a VO are reviewed. 1 Science Requirements for Data Mining What is data mining and why is applicable to scientific research? Data mining is defined as an information extraction activity whose goal is to discover hidden facts contained in databases. Data mining has taken the business community by storm and there is consequently now a vast array of resources and research techniques available for exploitation by the scientific communities. It is useful therefore to examine a categorization of data mining thrusts and their sub-components, since these are likewise applicable to the scientific exploration of large astronomical databases. Data mining is used to find patterns and relationships in data by using sophisticated techniques to build models – abstract representations of reality. A good model is a useful guide to understanding that reality and to making decisions. There are two main types of data mining models: descriptive and predictive. Descriptive models describe patterns in data and are generally used to create meaningful subgroups or clusters. Predictive models are used to forecast explicit values, based upon patterns determined from known results. There is another differentiation of data mining into two categories that we find particularly appropriate to knowledge discovery in large astronomical databases: event-based mining and relationship-based mining. At the risk of trivializing some fairly sophisticated techniques, we classify event-based mining scenarios into four orthogonal categories: • Known events / known algorithms – use existing physical models (descriptive models) to locate known phenomena of interest either spatially or temporally within a large database. • Known events / unknown algorithms – use pattern recognition and clustering properties of data to discover new observational (in our case, astrophysical) relationships among known phenomena. • Unknown events / known algorithms – use expected physical relationships (predictive models) among observational parameters of astrophysical phenomena to predict the presence of previously unseen events within a large complex database.